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Summary 

In the context of an emerging national focus on evaluating school personnel, the 2009 federal 
State Fiscal Stabilization Fund (SFSF) program requires that states collect data from all local 
education agencies about their practices for evaluating teachers and principals. In response, 
the California Department of Education designed and administered the California Teacher 
and Principal Evaluation Survey, the state s first comprehensive data collection effort focused 
on teacher and principal evaluation. More than 99 percent (1,482) of the state’s 1,490 local 
education agencies* returned the survey during the summer of 2010. 

Because the SFSF program did not require states to produce a summary report of the find- 
ings of their data collection, the California Department of Education has not aggregated the 
survey responses, instead posting them only by individual local education agency (California 
Department of Education 2011). Both the California Department ofEducation and the Inte- 
grated Leadership Development Initiative, a California cross- agency collaboration focused 
on improving school and district leadership, requested an analysis and summary report of the 
survey results not only to inform their work but also to help other California decisionmakers 
interested in reform of teacher and principal evaluation. In addition, they wanted to know 
whether the educator evaluations of districts and direct-funded charter schools differ in their 
consideration of student achievement outcomes or student growth data and in how evalua- 
tion results are used. 

Four research questions guided this analysis of the California survey data: 

• Flow did local education agencies describe their teacher and principal evaluation 
systems? 

• To what extent did local education agencies report that student achievement out- 
comes or student growth data were used in evaluating the performance of teachers 
and principals? Flow did the responses of districts differ from those of direct-funded 
charter schools? 

• To what extent did local education agencies report using evaluation results to inform 
personnel decisions for teachers and principals? Flow did the responses of districts 
differ from those of direct-funded charter schools? 

• To what extent did local education agencies report using evaluation results to distin- 
guish teachers and principals across multiple rating categories? 

The key findings indicate that: 

• Sixty-one percent of the 1,482 responding local education agencies indicated that 
their teacher evaluation systems are based on the California Standards for the Teach- 
ing Profession. 

• Forty-one percent reported that their local school board approves their teacher evalu- 
ation system; 64 percent reported that their local school board approves their princi- 
pal evaluation system. 

• For teacher evaluation, 57 percent reported using student achievement outcomes 
or growth data as partial or primary evidence; for principal evaluation, 79 percent 
reported using these data. 
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• Compared with responding districts, direct-funded charter schools reported greater 
use of student achievement or growth results as partial or primary evidence for edu- 
cator evaluation. The differences were more pronounced in teacher evaluation: 82 
percent of responding charter schools reported such use of student achievement or 
growth results, compared with 45 percent of districts. For principal evaluation, the 
figure was 85 percent of charter schools and 76 percent of districts. 

• Local education agencies reported using evaluation results in a variety of ways. They 
reported using results more often for high-stakes decisions about removal and reten- 
tion and less often for decisions about compensation and promotion, particularly for 
teachers. Teacher evaluation results were reportedly used as partial or primary evi- 
dence for removal decisions in 96 percent of local education agencies, for retention 
decisions in 93 percent, for promotion decisions in 54 percent, and for compensation 
decisions in 20 percent. Principal evaluation results were used as partial or primary 
evidence for removal decisions in 96 percent of local education agencies, for retention 
decisions in 94 percent, for promotion decisions in 67 percent, and for compensation 
decisions in 37 percent. 

• A larger percentage of direct-funded charter schools (27 percent) than of districts (18 
percent) reported using teacher evaluation results as the primary basis for decisions 
about professional development, promotion (17 percent versus 6 percent), and com- 
pensation (10 percent versus 1 percent). Both had a similar percentage report using 
evaluations in decisions on retention (41 percent in both) and removal (41 percent in 
both). 

• Differences between the two local education agency types in the use of principal 
evaluations were less uniform. A larger percentage of direct-funded charter schools 
(9 percent) than of districts (3 percent) reported using principal evaluation results 
as the primary basis for compensation decisions. However, a smaller percentage of 
direct-funded charter schools than of districts reported using principal evalua- 
tion results as the primary basis for decisions related to removal (36 percent versus 
43 percent), retention (35 percent versus 40 percent), and professional development 
(19 percent versus 24 percent). Both direct-funded charter schools and districts had 
a similar percentage report using evaluations in promotion decisions for principals 
(13 percent and 14 percent). 

• More than two-thirds of local education agencies reported having two or three per- 
formance rating levels for their teachers (37 percent had two levels, and 35 percent 
had three) and principals (40 percent had two levels, and 30 percent had three). Local 
education agencies with two rating levels reported that 98 percent of their teachers 
and 83 percent of their principals were rated in the highest category; agencies with 
three rating levels reported that 91 percent of their teachers and 98 percent of their 
principals were rated in the highest category. 

April 2012 
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Why this brief? 

Recently, the Obama administration and large 
philanthropic and research organizations have 
raised public awareness about what many con- 
sider to be poor teacher and principal evaluation 
practices in some of the country’s school districts. 
For example, recent federal guidance states that 
any School Improvement Grant-funded dis- 
trict implementing the school “transformation” 
model option must adopt “rigorous, transparent, 
and equitable evaluation systems for teachers and 
principals that take into account data on student 
growth as a significant factor” (U.S. Department 
of Education 2010, p. 26). Conversations about 
teacher and principal evaluation also are taking 
place in California, where the state’s largest dis- 
trict (Los Angeles Unified School District) has 
begun to overhaul its personnel evaluation sys- 
tem and the state legislature is considering new 
statewide policies for educator evaluation. 

In this context, the 2009 federal State Fis- 
cal Stabilization Fund (SFSF) program requires 
that states collect data from all local education 
agencies on their teacher and principal evalu- 
ation practices. In response, the California 
Department of Education designed and admin- 
istered the 2010 California Teacher and Princi- 
pal Evaluation Survey, the state’s first compre- 
hensive data collection focused on these issues. 
More than 99 percent (1,482) of the state’s 
1,490 local education agencies — districts^ and 
direct-funded charter schools^ — returned the 
survey during the summer of 2010. The data 
from this survey thus offer a timely perspec- 
tive on how California local education agencies 
evaluate their teachers and principals. 

The survey addresses the key issues noted in 
the SFSF requirements, including: 

• Flow local education agencies describe 
their teacher and principal evaluation 
systems, including how the systems 
were approved (whether by local board 
or through collective bargaining^), and 



the extent to which the teacher systems 
are based on professional standards. 

• The use of evaluation results in per- 
sonnel decisions — the extent to which 
teacher and principal evaluation results 
are used to inform development, com- 
pensation, promotion, retention, and 
removal decisions. 

• The role of student achievement out- 
comes or growth data in evaluating 
teachers and principals. 

• Ratings of teachers and principals 
based on evaluation results, including 
the numbers and names of the perfor- 
mance ratings or levels and the number 
of individuals rated at each level in the 
2009/ 10 school year. 

Because the SFSF program does not require 
states to produce a summary report of their 
survey findings, California’s survey responses 
have not been aggregated but have instead been 
posted by local education agency (California 
Department of Education 2011). Both the 
California Department of Education and the 
Integrated Leadership Development Initiative 
(ILDI),^ a California cross- agency collaboration 
focused on improving school and district leader- 
ship since 2005, requested an analysis and sum- 
mary report of the survey results. In addition to 
an overall analysis, they wanted to know whether 
districts and direct-funded charter schools dif- 
fer in how their evaluations consider student 
achievement or growth data and in how evalua- 
tion results are used. The two organizations indi- 
cated that such an analysis would help other Cal- 
ifornia decisionmakers interested in teacher and 
principal evaluation reform as well informing 
their own work. (Appendix A briefly summa- 
rizes the literature on teacher and principal eval- 
uation practices nationally and in California.) 

Four research questions guided this analysis 
of the California Teacher and Principal Evalu- 
ation Survey data. Two questions relate to local 
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education agency respondents in general, and 
two investigate reported differences between 
districts (including county offices of education) 
and direct-funded charter schools: 

• How did local education agencies 
describe their teacher and principal 
evaluation systems? 

• To what extent did local education 
agencies report that student achieve- 
ment outcomes or student growth data 
were used in evaluating the perfor- 
mance of teachers and principals? How 
did the responses of districts differ from 
those of direct-funded charter schools? 

• To what extent did local education 
agencies report using evaluation results 
to inform personnel decisions for 
teachers and principals? How did the 
responses of districts differ from those 
of direct-funded charter schools? 

• To what extent did local education 
agencies report using evaluation results 
to distinguish teachers and principals 
across multiple rating categories? 

A series of descriptive analyses of the Cali- 
fornia Teacher and Principal Evaluation Survey 
were conducted to explore these questions. Fre- 
quencies of response choices for all items were 
calculated, and cross-tabulations were prepared 
to investigate differences in the distributions 
of responses between direct-funded charter 
schools and districts (see box 1 and appendix B 
for study methodology). 

Study findings 

The results of descriptive analyses of the sur- 
vey data are reported in this section by study 
question. 

How local education agencies in California 
described their teacher and principal 
evaluation systems 

Sixty-one percent (903) of the 1,482 respond- 
ing local education agencies indicated that 
their teacher evaluation systems are based on 
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the California Standards for the Teaching 
Profession (figure 1). Forty-nine percent (726) 
indicated that their systems are determined 
through collective bargaining, and 41 percent 
(615) described them as local board-approved. 
For principal evaluations, 64 percent (950) of 
respondents reported that their systems are 
local board-approved (figure 2) and 3 percent 
(45) that they are determined through collective 
bargaining; 39 percent (578) responded “other.” 
These responses were elicited through two 
mark-all-that- apply survey items that prompted 
local education agencies to indicate the char- 
acteristics that “best describe” their evaluation 
systems for teachers and principals. Response 

FIGURE 1 

Types of teacher evaluation systems 
reported by local education agencies 
on the 2010 California Teacher and 
Principal Evaluation Survey 

Percent 




Local board- Bargained Evaluation Other 
approved evaluation system based evaluation 
evaluation system on the system 

system (726) California (143) 

(615) Standards for 

theTeaching 
Profession 
(903) 

Note: Numbers in parentheses are number of 
responses. The results reflect 2,387 responses from 
1,482 local education agencies. Because respon- 
dents could select any combination of the listed 
options, the number of overall responses is 
greater than the number of respondents and the 
percentages sum to more than 100. 

Source: Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 
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BOX 1 

Study methodology 

The 2010 California Teacher and Prin- 
cipal Evaluation Survey included 15 
closed-response questions about local 
education agency systems of teacher 
and principal evaluation. The survey 
also featured two open-response ques- 
tions on performance levels/ratings 
assigned to teachers and principals in 
the evaluation systems. Appendix B 
describes the methods used to clean, 
tabulate, and analyze the survey data; 
appendix C presents an additional set 
of data tables; appendix D contains 
the survey questions in full detail; and 
appendix E shows samples of how the 
questions appeared online. 

In addition to the 15 survey items, 
the online survey form asked a few 
initial questions. Local education 
agencies were asked to identify their 
county and to specify their type: 
school district, county office of 
education, or direct-funded charter 
school. Among responses, 1,013 local 
education agencies self-identified as 



districts (or county offices coded as 
districts) and 469 as direct-funded 
charter schools. (According to the 
California Department of Educa- 
tion, 71 percent of charter schools 
are directly funded by the state.) 

On May 25, 2010, a link to the online 
survey was sent to all California local 
education agencies through their State 
Eiscal Stabilization Eund coordina- 
tor, who forwarded it to appropriate 
district representative. Surveys were 
to be completed by June 11, 2010. 
However, because the survey system 
enabled partial responses to be sub- 
mitted, the California Department of 
Education had to request completion 
of missing items. Letters requesting 
missing information were sent on 
July 15, 2010 (appendix E ) and again 
on September 24, 2010 (appendix 
G). Eollow-up phone calls were also 
made. Thus the system was open for 
data submission and changes from 
May 25-September 30, 2010. The 
final survey response rate was 99.5 
percent, with 1,482 of the state’s 1,490 
local education agencies responding. 



The study team conducted a series 
of descriptive analyses, calculat- 
ing frequencies of response choices 
for all items and preparing cross- 
tabulations to investigate differences 
in the distributions of responses of 
direct-funded charter schools and 
districts for two research questions, 
for the two open-response questions 
that asked about the performance 
levels/ratings assigned to teachers and 
principals, the study team reviewed 
and reclassified the response data as 
necessary to report only ordinal scales 
of performance in keeping with the in- 
structions for the survey.' Researchers 
excluded from the analysis entries that 
clearly did not follow an ordinal logic 
or for which the order could not be 
determined because the levels/ratings 
were not interpretable. A complete 
description of the coding and classifi- 
cation procedures is in appendix B. 

Note 

1. The survey instructions specified that 
the highest rating must be assigned 
ranking order 1, the next highest 2, and 
so on (see appendix D). 



choices included local board-approved system, 
bargained evaluation system, and other evalu- 
ation system (describe). For teacher evaluation 
systems, the survey offered the additional choice 
of an evaluation system based on the California 
Standards for the Teaching Profession. Because 
respondents could select any combination of the 
listed options, the number of overall responses is 
greater than the number of respondents. 

Use of student achievement outcomes as a 
performance evaluation criterion 

The SFSF guidelines required that states report 
for each local education agency “whether the 



systems used to evaluate the performance of 
teachers include student achievement out- 
comes or student growth data as an evalua- 
tion criterion.” There is a parallel requirement 
for principal evaluation systems. Local educa- 
tion agencies reported most frequently that 
student achievement data are used as “partial 
evidence” in their teacher and principal sys- 
tems (figure 3), with use higher for principal 
evaluations (70 percent; 1,033 respondents) 
than for teacher evaluations (53 percent; 782 
respondents). Forty-three percent (638) stated 
that student achievement data are not used as a 
criterion in teacher evaluation, and 21 percent 
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FIGURE 2 

Types of principal evaluation systems 
reported by local education agencies 
on the 2010 California Teacher and 
Principal Evaluation Survey 

Percent 
75 



50 



25 



0 

Local board- Bargained Other 

approved evaluation evaluation 

evaluation system system 

system (45) (578) 

(950) 

Note: Numbers in parentheses are number of 
responses. The results reflect 1,573 responses from 
1,482 local education agencies. Because respon- 
dents could select any combination of the listed 
options, the number of overall responses is 
greater than the number of respondents and the 
percentages sum to more than 100. 

Source; Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 




(316) indicated that these data are not used 
in principal evaluation. Four percent (62) of 
local education agencies indicated that student 
achievement data are used as the primary basis 
of decisions in teacher evaluation; 9 percent 
(133) of respondents reported these data being 
used similarly in principal evaluation. 

Difference between districts and direct- 
funded charter schools in use of student 
achievement outcomes as a performance 
evaluation criterion 

Cross-tabulation was used to explore differ- 
ences in responses on use of student achieve- 
ment outcomes in performance evaluations 
by local education agency type — districts 



EIGURE3 

Use of student achievement 
outcomes to evaluate teacher and 
principal performance as reported 
by local education agencies on the 
2010 California Teacher and Principal 
Evaluation Survey 



Percent 



■ Teachers ■ Principals 




Not used for Used as Used as 

this purpose partial evidence primary basis 
for decisions 



Note: Numbers in parentheses are number of re- 
sponses. The results reflect responses from 1,482 local 
education agencies. 

Source; Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 



(including county education offices) and direct- 
funded charter schools.^ For teacher evalua- 
tion, 82 percent (386) of direct-funded charter 
schools reported using student achievement 
outcomes as partial or primary evidence in 
teacher evaluation compared with 45 percent 
(458) of responding districts (figure 4). 

For principal evaluation, 68 percent (693) 
of responding districts and 72 percent (340) of 
responding direct-funded charter schools indi- 
cated that they use student achievement out- 
comes as partial evidence in principal evaluations. 
Seven percent (74) of responding districts and 
13 percent (159) of responding charter schools 
reported using such data as the primary basis for 
decisions. Twenty-four percent (246) of districts 
and 15 percent (70) of charter schools reported 
that these data were not used for this purpose. 
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FIGURE4 

Use of student achievement 
outcomes to evaluate teacher 
performance as reported by districts 
and direct-funded charter schools 
on the 2010 California Teacher and 
Principal Evaluation Survey 



FIGURES 

Use of student achievement 
outcomes to evaluate principal 
performance as reported by districts 
and direct-funded charter schools 
on the 2010 California Teacher and 
Principal Evaluation Survey 



Percent ■ District ■ Direct-funded charter school 



Percent ■ District ■ Direct-funded charter school 





Not used for Used as partial Used as 

this purpose evidence primary basis 

for decisions 



Note: Numbers in parentheses are number of 
responses. The results reflect responses from 1,013 
districts and 469 charter schools. Percentages may 
not sum to 100 because of rounding. 

Source.- Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 



Note: Numbers in parentheses are number of 
responses. The results reflect responses from 1,013 
districts and 469 charter schools. Percentages may 
not sum to 100 because of rounding. 

Source.- Authors' analysis of data from California 
Teacher and Principal Evaluation Survey provided to 
REL West by CDE. 



Use of evaluation results to inform personnel 
decisions 

Survey items also probed how evaluation results 
were used to inform the following personnel 
decisions (listed but not defined): removal, 
retention, professional development, promo- 
tion, and compensation. For teachers, local edu- 
cation agencies reported relying most heavily 
on evaluation results for decisions of removal 
and retention (figure 6). Forty-one percent (603 
for removal and 611 for retention) of respond- 
ing local education agencies used evaluation 
results as the primary basis for teacher-related 
decisions in these two high-stakes areas; less 
than 8 percent reported that teacher evalua- 
tion results are not used for this purpose — 4 
percent (58) for removal decisions and 7 percent 



(97) for retention decisions. Seventy-nine per- 
cent (1,174) of responding local education 
agencies indicated that they do not use evalu- 
ation results for teacher compensation deci- 
sions, while 3 percent (51) reported that teacher 
evaluation results are the primary basis for such 
decisions. For promotion decisions, 47 percent 
(694) reported that they do not use evalua- 
tion results for this purpose, while 10 percent 
(142) reported that they are the primary basis 
for such decisions. Seventy-two percent (1,073) 
indicated that evaluations offer partial evidence 
for teacher professional development decisions, 
while 20 percent (302) reported that evaluation 
results are the primary basis for such decisions. 

Results were similar for use of principal eval- 
uation results (figure 7). Key differences between 
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FIGURE6 

Use of teacher evaluation system results to make 
personnel decisions as reported by local education 
agencies on the 2010 California Teacher and 
Principal Evaluation Survey 



I Not used for Used as ■ Used as primary 

this purpose partial evidence basis for decisions 




Removal Retention Professional Promotion Compensation 
development 



Wofe; Numbers in parentheses are number of responses. The results 
reflect responses from 1,482 local education agencies. For each category 
of personnel decision, the figure identifies the percentage and number of 
responding local education agencies that selected each response choice 
(not used for this purpose, used as partial evidence, and used as primary 
basis for decisions). Percentages may not sum to 1 00 because of rounding. 



FIGURE? 

Use of principal evaluation system results to make 
personnel decisions as reported by local education 
agencies on the 2010 California Teacher and 
Principal Evaluation Survey 



Percent 



■ Not used for Used as ■ Used as primary 

this purpose partial evidence basis for decisions 



100 



75 



50 



25 



0 




Removal Retention Professional Promotion Compensation 
development 



Wofe.- Numbers in parentheses are number of responses. The results 
reflect responses from 1,482 local education agencies. For each category 
of personnel decision, the figure identifies the percentage and number of 
responding local education agencies that selected each response choice 
(not used for this purpose, used as partial evidence, and used as primary 
basis for decisions). Percentages may not sum to 1 00 because of rounding. 



Source; Authors' analysis of data from California Teacher and Princi- 
pal Evaluation Survey data provided by the California Department of 
Education. 



Source; Authors' analysis of data from California Teacher and Princi- 
pal Evaluation Survey data provided by the California Department of 
Education. 



teacher- and principal-focused responses related 
to the use of evaluation results in promotion and 
compensation decisions. A higher proportion of 
local education agencies reported using results 
from principal evaluations than from teacher 
evaluations as partial or primary evidence in 
decisions related to promotion — 67 percent 
(1,000) for principals versus 53 percent (788) for 
teachers — or compensation — 37 percent (551) 
versus 21 percent 308). 

Difference between districts and direct- 
funded charter schools in use of evaluation 
results to inform personnel decisions 

Survey results indicate that a larger percent- 
age of direct-funded charter schools than of 
districts reported using teacher evaluation 
results as the primary basis for professional 



development (27 percent versus 18 percent), 
promotion (17 percent versus 6 percent), and 
compensation (10 percent versus 1 percent) 
decisions (figure 8). Both local education 
agency types had a similar percentage report 
using evaluation results as the primary basis 
for decisions related to removal (41 percent for 
both) and retention (41 percent for both). 

Responding districts and direct-funded 
charter schools had a similar percentage report 
using principal evaluation results as the pri- 
mary basis for promotion decisions (13 percent 
versus 14 percent; figure 9). However, a smaller 
percentage of direct-funded charter schools 
than of districts reported using principal evalu- 
ation results as the primary basis for removal 
(36 percent versus 43 percent), retention (35 
percent versus 40 percent), and professional 
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FIGURES 

Percentage of personnel decisions 
based primarily on teacher evaluation 
results as reported by districts and 
direct-funded charter schools on the 
2010 California Teacher and Principal 
Evaluation Survey 



FIGURE 9 

Percentage of personnel decisions 
based primarily on principal 
evaluation results as reported by 
districts and direct-funded charter 
schools on the 2010 California Teacher 
and Principal Evaluation Survey 



Percent ■ District ■ Direct-funded charter school 



Percent ■ District ■ Direct-funded charter school 




Removal 



Retention Professional Promotion Compensation 
deveiopment 



60 



43 

1435 ) 




Removai Retention Professionai Promotion Compensation 
deveiopment 



Note: Numbers in parentheses are number of re- 
sponses. The results reflect responses from 1,013 dis- 
tricts and 469 direct-funded charter schools. For each 
category of personnel decision, the figure identifies 
the percentage and number of responding districts 
and direct-funded charter schools selecting "used 
as primary basis for decisions." Because this figure 
presents data from a survey question with multi- 
ple related parts, the number of overall responses 
is greater than the number of respondents, and 
the percentages sum to more than 100. 

Source.- Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 



Note: Numbers in parentheses are number of re- 
sponses. The results reflect responses from 1,013 dis- 
tricts and 469 direct-funded charter schools. For each 
category of personnel decision, the figure identifies 
the percentage and number)of responding districts 
and direct-funded charter schools selecting "used 
as primary basis for decisions." Because this figure 
presents data from a survey question with multi- 
ple related parts, the number of overall responses 
is greater than the number of respondents, and 
the percentages sum to more than 100. 

Source.- Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 



development (19 percent versus 24 percent) 
decisions. A larger percentage of direct-funded 
charter schools than of districts reported using 
principal evaluation results as the primary basis 
for compensation decisions (9 percent versus 
3 percent). 

Use of evaluation results to distinguish teachers 
or principals across multiple rating categories 

The final survey questions asked for the 
names of the summative performance ratings 
or levels that teachers and principals receive 
in evaluations. For example, the evaluation 



system in one district might have two perfor- 
mance ratings or levels, such as “satisfactory” 
and “unsatisfactory,” whereas another district 
might have three performance ratings or lev- 
els, such as “highly effective,” “effective,” and 
“ineffective.” Respondents were also asked 
to provide the number of teachers that were 
rated in each category at each school site 
within the local education agency, as well as 
the number of the principals rated in each 
category. The survey had provisions for up to 
eight rating categories in each school or local 
education agency. 
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The teacher performance ratings are 
reported at the school level, because local edu- 
cation agencies reported teacher ratings for 
each school and the results indicated that the 
number of performance levels varied across 
schools in some local education agencies; the 
principal performance ratings are reported at 
the local education agency level (table 1). The 
majority of schools and local education agencies 
had two or three performance rating levels for 
teachers (37 percent had two levels, and 35 per- 
cent had three) and principals (40 percent had 
two levels, and 30 percent had three). Another 
9 percent of schools had just one rating level for 
teachers, and 8 percent of local education agen- 
cies had just one for principals. 

For schools with two teacher rating levels, 
98 percent of teachers were rated in the high- 
est category; for schools with three levels, 91 
percent were rated in the highest category 
(table 2).^ The distribution of teachers across 
rating levels was wider in schools with four or 
more levels, although fewer schools use this 
many (see table 1). 



For local education agencies with two rat- 
ing levels for principals, 97 percent of princi- 
pals were rated in the highest category; for local 
education agencies with three levels, 83 percent 
were rated in the highest category (table 3). For 
local education agencies with three rating lev- 
els, the reported spread was wider for principals 
than for teachers (83 percent of principals were 
rated in the highest level compared with 91 per- 
cent of teachers). 

Conclusion and implications 

Reforms of teacher and principal evaluation 
systems are now under way in California, as 
the Los Angeles Unified School District moves 
to reform its entire system of personnel evalu- 
ation and support, the state legislature consid- 
ers revisions to state policy on educator evalu- 
ation, and federal pressures for reform mount, 
through both the federal School Improvement 
Grant program and anticipated reforms within 
the upcoming reauthorization of the Elemen- 
tary and Secondary Education Act. Before the 
California Teacher and Principal Evaluation 



TABLE 1 

Number and percentage of schools or local education agencies by number of 
teacher and principal performance ratings as reported by local education agencies 
on the 2010 California Teacher and Principal Evaluation Survey 



Number of 
performance 
ratings or levels 


Teacher evaluation systems 


Principal evaluation systems 


Number of 
schools 


Percent of 
schools 


Number of 
local education 
agencies 


Percent of local 
education 
agencies 


1 


822 


9 


111 


8 


2 


3,581 


37 


565 


40 


3 


3,324 


35 


415 


30 


4 


1,553 


16 


226 


16 


5 


171 


2 


81 


6 


6 


112 


1 


a 


a 



Note: See appendix B for data and methods. Teacher evaluation levels were reported at the school level in the survey, 
and some districts reported variation in the number of rating levels across schools; principal evaluation levels were 
reported only at the district level. 

a. To protect confidentiality, results are not displayed for cells with three or fewer responses. 

Source; Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California 
Department of Education. 
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Conclusion and implications 



TABLE 2 

Distribution of teachers across teacher performance ratings or levels as reported by local education agencies 
on the 2010 California Teacher and Principal Evaluation Survey 



Number 
of levels 










Rating level (1 is highest, 6 is lowest) 






Number of 
teachers 


1 

Number 


Percent 


2 

Number 


3 

Percent Number 


Percent 


4 

Number 


5 

Percent Number 1 


6 

Percent Number Percent 


1 


18,403 


100 


— 


— 


— 


— 


— 


— 


— 


— — — 


18,403 


2 


94,794 


98 


1,887 


2 


— 


— 


— 


— 


— 


— — — 


96,681 


3 


87,162 


91 


7,474 


8 


1,123 


1 


— 


— 


— 


— — — 


95,759 


4 


16,318 


42 


20,221 


52 


1,735 


4 


555 


1 


— 


— — — 


38,829 


5 


1,096 


25 


2,350 


55 


728 


17 


90 


2 


26 


1 — — 


4,290 


6 


1,610 


52 


1,021 


33 


325 


10 


63 


2 


52 


2 21 1 


3,092 



— is not applicable. 

/Vote.- The results reflect responses by local education agencies for 9,290 schools with 257,054 teachers. 

Source.- Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California Department of Education. 



TABLE 3 

Distribution of principals across performance ratings or levels as reported by local education agencies on the 
2010 California Teacher and Principal Evaluation Survey 



Number 
of levels 










Rating level (1 is highest, 6 is lowest) 




Number of 
principals 


1 

Number 


Percent 


2 

Number 


3 

Percent Number 


Percent 


4 

Number 


Percent 


5 

Number 


6 

Percent Number Percent 


1 


316 


100 


— 


— 


— 


— 


— 


— 


— 


— — — 


316 


2 


3,737 


97 


97 


3 


— 


— 


— 


— 


— 


— — — 


3,834 


3 


2,056 


83 


355 


14 


66 


3 


— 


— 


— 


— — — 


2,477 


4 


728 


42 


865 


50 


103 


6 


30 


2 


— 


— — — 


1,726 


5 


81 


21 


201 


53 


74 


19 


16 


4 


8 


2 — — 


380 


6 


16 


31 


6 


11 


16 


31 


11 


21 


a 


a a a 


a 



— is not applicable. 

Note: Because of missing data, the results reflect responses by 1,367 local education agencies for 8,782 principals. See appendix B for details on methodology, 
a. To protect confidentiality, results are not displayed for cells with three or fewer responses orfor cells that could be used to calculate values for such cells. 
Source.- Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California Department of Education. 



Survey administered in 2010, very little infor- 
mation was available on evaluation practices 
in the state. This analysis of these survey data, 
though limited by the nature of the survey ques- 
tions, provides a first look at the status of educa- 
tor evaluation in virtually every California local 
education agency. Survey results indicated that: 
• Sixty-one percent of the 1,482 
responding local education agencies 
indicated that their teacher evaluation 



systems are based on the California 
Standards for the Teaching Profession. 

• Forty-three percent of responding 
local education agencies stated that 
student achievement data are not used 
as a criterion in teacher evaluation, and 
21 percent indicated that these data 
are not used in principal evaluations.^ 

• Direct-funded charter schools 
reported greater reliance on student 
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achievement results than did districts. 
The differences were more pronounced 
in reported teacher evaluation prac- 
tices than in reported principal evalu- 
ation practices. 

• Local education agencies reported 
using evaluation results in a variety 
of ways. Results were reportedly used 
most frequently in high-stakes deci- 
sions of removal and retention and less 
used in compensation and promotion 
decisions, particularly for teachers. 

• Compared with responding districts, 
a higher percentage of direct-funded 
charter schools than of districts 
reported using teacher evaluation 
results as the primary basis for profes- 
sional development, promotion, and 
compensation decisions. 

• For principals, a higher percentage of 
direct-funded charter schools than of 
districts reported using evaluations to 
inform compensation decisions; how- 
ever, districts reported using evalua- 
tions to inform removal, retention, and 
professional development decisions. 

• Over two-thirds of local education 
agencies reported having two or three 
performance rating levels for their 
teachers (37 percent had two levels, 
and 35 percent had three) and princi- 
pals (40 percent had two levels, and 30 
percent had three). For local education 
agencies reporting two rating levels, 
98 percent of teachers and 97 percent 



Study limitations 



of principals were rated in the high- 
est category. For agencies reporting 
three levels, 91 percent of teachers and 
83 percent of principals were rated in 
the highest category. 

These findings represent a first step toward 
building a better understanding of the teacher 
and principal evaluation practices in place in 
California. 

Study limitations 

The conclusions that can be drawn from this 
analysis are limited by the content and struc- 
ture of the survey questions and by the self- 
reported nature of the data. For example, the 
mark-all-that- apply format of questions 1.1 
and 1.2 rendered interpretation a challenge, 
especially in light of the many possible combi- 
nations of responses, some of which were not 
focused on parallel aspects of the evaluation 
systems. In addition, information reported on 
the survey may be incorrect to the extent that 
the respondents did not understand the ques- 
tions or misreported responses, as key terms 
and categories were undefined. No data veri- 
fication process was completed, and no other 
data were used to triangulate the survey data. 
Finally, no information was available on prac- 
tices in district- or county-funded charter 
schools; the only accessible data on charter 
practices came from charter respondents who 
self-identified as directly funded by the state 
when completing the survey. A more complete 
summary of the challenges related to survey 
interpretation is available in appendix B. 
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Appendix A 

The literature on teacher and 
principal evaluation practices 

This appendix briefly summarizes the literature 
on teacher and principal evaluation practices 
nationally and in California. 

Teacher and principal evaluation practices 
nationally and in other regions 

There have been only a few studies of teacher 
or principal evaluation practices.^ Regional 
Educational Laboratory Midwest reviewed 
district policies for teacher evaluation in the 
Midwest Region (Brandt et al. 2007), and Hazi 
and Arredondo-Rucinski (2009) conducted a 
50-state review of teacher evaluation policies. 
Brandt et al. found that less than half of dis- 
tricts set requirements for the use of evaluation 
results in personnel decisions; of those that 
did, 45 percent stated that the results would be 
used to inform professional development. Hazi 
and Arredondo-Rucinski found that the most 
common state -level efforts were related to the 
adoption of indicators and standards defin- 
ing effective teaching (20 states) and requiring 
training for evaluators (20 states); less common 
state requirements were related to the use of 
peer review and portfolios (12 states) and use of 
student achievement data in evaluating teach- 
ers (12 states). 

Several policy reports have described prob- 
lems associated with current evaluation prac- 
tices (see Toch and Rothman 2008; Weisberg 
et al. 2009; Daley and Kim 2010). In particu- 
lar, Weisberg et al. (2009) found that teacher 
evaluations tend to be infrequent, results fail to 
differentiate among teachers, and student aca- 
demic progress is rarely considered.^® Several 
reviews of teacher evaluation systems have dis- 
cussed the merits and challenges of incorporat- 
ing student achievement data into evaluation 
practices (for example, Goe, Bell, and Little 
2008; National Research Council and National 
Academy of Education 2010; Steele, Hamilton, 
and Stecher 2010). Much of the other empirical 



literature related to evaluation systems seeks to 
validate particular evaluation instruments (see, 
for example, Kimball, White, and Milanowski 
2004; Kimball and Milanowski 2009). 

There are fewer published empirical stud- 
ies of evaluation practices for principals than 
for teachers. Much of the literature focuses on 
instrument validation rather than implemen- 
tation (see, for example, Condon and Clifford 
2010). One recent large-scale study of national 
principal evaluation practices found that most 
lacked usefulness, psychometric strength, and 
instrument accuracy (Goldring et al. 2009).'* 
Although limited by a low response rate (22 
percent), another recent study found that assis- 
tant superintendents tended to serve as pri- 
mary evaluators for principals; the study also 
highlighted problems in training evaluators 
and calibrating their ratings (Kimball, Hene- 
man, and Milanowski 2007).'^ 

Teacher and principal evaluation practices in 
California 

The California Education Code includes gen- 
eral provisions about personnel evaluations, 
including their minimum frequency, as well 
as a provision that student achievement data 
must be used in evaluations.'^ However, nei- 
ther existing data nor research reports fully 
reveal the extent to which these provisions are 
reflected in local education agency practices. 

The limited number of studies on teacher 
evaluation practices in California offers find- 
ings similar to those from the national litera- 
ture.'^ For example, a review of seven districts’ 
teacher evaluation practices conducted in 2007 
by the Center for the Future of Teaching and 
Learning found evidence of infrequent class- 
room observations, an emphasis on classroom 
management skills rather than student out- 
comes, and limited use of information from 
the evaluations (Wechsler et al. 2007). A New 
Teacher Project analysis of evaluation practices 
in the Los Angeles Unified School District in 
2009 found that less than 1 percent of teachers 
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in the district were rated “below standard” on 
their evaluation and that 63 percent of teachers 
surveyed felt they were provided with informa- 
tion or strategies to help develop the quality 
of their instruction (although a precise defini- 
tion of such strategies was not provided; New 
Teacher Project 2009).^^ 

There is comparatively less research on prin- 
cipal evaluation practices in California. A 1999 
study by Davis and Hensley found that 19 of 
20 northern California principals interviewed 
reported that their evaluations were purely 
summative rather than oriented toward provid- 
ing formative professional assistance. 

Evaluation in charter schools 

Several recent reports have examined evaluation 
practices in charter schools.'^ For example, in 
their examination of a random sample of 132 
charter schools across seven states, Podgursky 
and Ballou (2001) found that 8 percent had 



some sort of collectively bargained agreement 
and that more than 40 percent considered at 
least some measure of student achievement 
in their evaluation of teachers.'^ To examine 
charter school evaluation practices in more 
depth, Donaldson and Peske (2010) purposively 
selected five charter schools run by three estab- 
lished charter management organizations. They 
found that teachers in these charter schools 
viewed evaluation processes as more frequent 
and “more robust” (p. 10) than evaluation pro- 
cesses in their prior schools (whether charter or 
noncharter schools).'^ According to the authors, 
the charter school evaluations focused more on 
professional growth than on summative assess- 
ment, teachers tended to be engaged in criti- 
cally reviewing their own practice, and student 
performance played a “key role” (p. 14) in all 
five systems. However, because of the small and 
nonrepresentative sample of charter schools, the 
findings from this study are not generalizable. 
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Appendix B 

Study data and methods 

This appendix provides details on the study 
data and methods. 

Data 

The California Teacher and Principal Evalua- 
tion Survey designed by the California Depart- 
ment of Education and administered in 2010 
included 15 closed- response questions and 
2 open-response questions about the perfor- 
mance levels or ratings that current evalua- 
tion systems assign to teachers and principals. 
Appendix D contains the survey questions. 
Appendix E shows samples of how the ques- 
tions appeared on the online survey. 

Questions 1.1 and 1.2, which were mark- 
all-that- apply, asked respondents to indicate 
the descriptors that “best describe” their 
evaluation systems for teachers and for princi- 
pals. Response choices included “local board- 
approved system,” “bargained evaluation sys- 
tem,” and “other evaluation system (describe).” 
Item 1.1, concerned with teacher evaluation 
systems, included the additional choice of 
“evaluation system based on the California 
Standards for the Teaching Profession.” 

Item 2.1 asked about issues required in the 
2009 federal State Fiscal Stabilization Fund 
(SFSF) criteria, specifically: “describe, for each 
local educational agency (LEA) in the State, 
the systems used to evaluate the performance 
of teachers and the use of results from those 
systems in decisions regarding teacher devel- 
opment, compensation, promotion, retention, 
and removal.” Item 2.2 asked about the parallel 
requirement for principal systems. 

Item 3.1 was intended to elicit information 
to meet the SFSF requirement that the state 
report for each local education agency “whether 
the systems used to evaluate the performance of 
teachers include student achievement outcomes 
or student growth data as an evaluation crite- 
rion.” Item 3.2 asked about the parallel require- 
ment for principal systems. 
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Item 4.1 asked local education agencies to 
report on the extent to which the ratings prin- 
cipals receive were publicly reported. 

Items 5.1 and 5.2 elicited information to 
meet the SFSF requirements asking each local 
education agency “whose teachers receive per- 
formance ratings or levels through an evalua- 
tion system . . .” and “whose principals receive 
performance ratings or levels through an evalu- 
ation system” to provide “the number and per- 
centage (including numerator and denomina- 
tor) of principals rated at each performance 
rating or level.” These two open-response ques- 
tions specifically asked local education agencies 
to name the summative performance ratings 
or levels teachers and principals received (for 
example, “satisfactory” or “unsatisfactory”). 
Respondents were also asked to provide the 
number of teachers at each school site in the 
local education agency in each rating category 
and the number of principals at each rating cat- 
egory in each local education agency (but not 
at the school level). The survey allowed entry 
of up to eight rating categories for teachers 
and for principals in each school in each local 
education agency. For example, instructions for 
item 5.1 included (complete instructions are in 
appendix D): 

SFSF indicator (a)(4) requires that each 
LEA indicate the number and per- 
centage of teachers rated at each per- 
formance level or rating. To enter this 
information, begin by selecting “Man- 
age Teacher Categories.” Use the “Add a 
Category” button to enter the first rat- 
ing level of the teacher evaluation sys- 
tem. For example, this category might 
be named “Satisfactory.” The ranking 
order indicates the level of performance 
for that category. The highest rank, 

1, would be used to indicate the high- 
est level of performance a teacher can 
demonstrate. Click the “Save Changes” 
button. Continue to add categories 
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until the number and descriptions of 
categories exactly matches those used 
in the teacher evaluation system. After 
saving the changes, select the “School 
Level Data-Teachers” button. Enter the 
total numbers of teachers that are rated 
in each category for each school in the 
LEA. Include all teachers at the site 
regardless of the last date of evaluation. 

The sum of all categories should equal 
the total number of teachers at the site. 

The copy of the survey in appendix D shows 
items 5.1 and 5.2 as screenshots from a sample 
survey in which actual school names are listed 
in 5.1, as are sample performance level names 
(in 5.1, “satisfactory” and “unsatisfactory,” and 
in 5.2, “practice not consistent with standards” 
and “practice that exemplifies standards”). 

In addition to the main survey items, the 
online survey form asked a few initial ques- 
tions. Respondents were asked to select the 
name of their county and local education 
agency from drop-down lists. They were also 
asked to specify their local education agency 
type: school district, county office of education, 
or direct-funded charter school. In California, 
some charter schools are funded through dis- 
tricts or county offices, and others directly from 
the state. The survey respondents consisted of 
955 districts and 58 county offices, collectively 
referred to and analyzed in this study as 1,013 
“districts,” and 469 direct-funded charter 
schools. 

On May 25, 2010, a link to the online sur- 
vey was sent to all California local education 
agencies through each agency’s SFSF coordi- 
nator, who was then expected to forward the 
link to the district representative best able to 
respond. Respondents initially had until June 
11, 2010, to respond. However, according 
to the California Department of Education, 
because the survey system allowed respondents 
to submit only partial data, the department 
had to contact all local education agencies with 
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partial submissions and request completion 
of missing items. Letters were sent on July 15, 
2010 (appendix F), and again on September 24, 
2010 (appendix G). Phone calls were also made 
to all local education agencies that had not sub- 
mitted complete data. The system was open for 
data submission and changes from May 25 to 
September 30, 2010. The final survey response 
rate was 99.5 percent (1,482 of 1,490 local 
education agencies responded). At the time of 
writing, the study team had not received the 
list of nonresponding local education agencies 
(the California Department of Education had 
cleaned and prepared the dataset). 

Analysis 

After cleaning the data, the study team 
conducted a series of descriptive analyses, 
reporting on item frequencies for the closed- 
response items and running cross-tabulations 
to investigate differences in the responses of 
direct-funded charter schools and districts for 
research questions 2 and 3. 

To analyze the performance rating/level 
data entered in response to questions 5.1 and 
5.2, the study team began by reviewing the 
entries to ensure that they were all in the same 
ordinal sequence from highest to lowest, as 
specified in the survey instructions. Many were 
not, for a variety of reasons. Many were entered 
in reverse order (lowest to highest). Others did 
not follow an ordinal format — for example, 
they referred to names of standards or schools 
in the local education agency, or were other- 
wise uninformative. To address these issues, 
researchers classified each record into the cor- 
rect ordinal sequence whenever possible, and 
removed the entries that were not usable for 
other reasons. A detailed summary of these 
procedures is provided below. 

Teacher performance ratings/levels. The teacher 
ratings file began with 9,898 records and 
included 94 duplicate records that were iden- 
tified by reviewing county-district-school 
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codes. The duplicate records were removed 
and the remaining 9,804 were coded in the 
same manner as for the principal rating records. 
Those coded 9+ and removed from the analy- 
sis included names of standards, school names, 
statements of teacher qualifications (such as 
“highly qualified”), notes about whether a 
teacher had been observed or evaluated, entries 
of “not applicable,” entries where the ordering 
was not apparent, and other uninformative 
text. 

After the initial coding, the study team 
removed all records that were coded as 9+ in 
every rating category for a given record (92 
records) and removed all the records that 
reported no rating categories (149 records). 
Researchers then reduced the number of per- 
formance ratings/levels in 198 records where 
some but not all of the entries were coded with 
a 9-1- (see table B1 for a breakdown of recodes). 
With these classification and cleaning tasks 
completed, 9,563 records remained, including 
273 for which performance ratings/levels were 
entered but no corresponding teacher counts 
were entered. These 273 records were not 
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included in the analysis of teacher rating cate- 
gories, which left 9,290 records for the analysis 
of 257,054 teachers by rating category. 

Principal performance ratings/levels. The prin- 
cipal ratings file began with 1,460 records 
and included no duplicates. Researchers 
assigned a numerical value to each performance 
rating/level within each record, with “1” as the 
highest possible rating and “8” as the lowest. 
For example, a record with the three ranking 
categories labeled “meets expectations,” work 
progressing,” and “unsatisfactory” would have 
those categories rated 1, 2, and 3, respectively. 
Responses for which assignment was not pos- 
sible were coded “9-I-” (see table B2 for a break- 
down of recodes). These included entries that 
could not be ordered or were otherwise unin- 
formative because they were not performance 
ratings/levels at all but were rather job titles, 
standards, school names, or statements such as 
“evaluation in progress” or “principal was not 
evaluated.” 

After the initial coding, the study team 
removed all the records that were coded as 9-1- 



TABLE B1 

Recoding of teacher performance ratings/levels 



Number of 
categories after 






Original number of rating categories 








data cleaning 


0 


1 


2 


3 


4 


5 


6 


7 


8 


Total 


0 


149 


62 


5 


16 


4 


2 


2 


0 


1 


241 


1 


0 


813 


0 


4 


5 


0 


0 


0 


0 


822 


2 


0 


0 


3,575 


6 


0 


0 


0 


0 


0 


3,581 


3 


0 


0 


0 


3,272 


52 


0 


0 


0 


0 


3,324 


4 


0 


0 


0 


0 


1,431 


122 


0 


0 


0 


1,553 


5 


0 


0 


0 


0 


0 


167 


4 


0 


0 


171 


6 


0 


0 


0 


0 


0 


0 


107 


5 


0 


112 


Total 


149 


875 


3,580 


3,298 


1,492 


291 


113 


5 


1 


9,804 



Note: An example can help in interpreting this table. Column 3 indicates that there were originally 3,298 records with 
three rating categories. After the entries coded 9-t (entries that could not be ordered or that did not reflect perfor- 
mance ratings/levels) were deleted, 3,272 entries with three rating categories remained, while 6 of the original 3,298 
were reclassified as having two categories, 4 were reclassified as having one, and 16 were reduced to zero. 

Source; Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California 
Department of Education. 
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TABLE B2 

Recoding of principal performance ratings/levels 



Number of 
categories after 




Original number of rating categories 




Total 


data cleaning 


1 


2 


3 


4 


5 


6 


8 


0 


35 


9 


3 


4 


4 


2 


1 


58 


1 


111 


0 


0 


0 


0 


0 


0 


111 


2 


0 


565 


0 


0 


0 


0 


0 


565 


3 


0 


0 


406 


9 


0 


0 


0 


415 


4 


0 


0 


0 


215 


10 


1 


0 


226 


5 


0 


0 


0 


0 


79 


2 


0 


81 


6 


0 


0 


0 


0 


0 


3 


0 


3 


8 


0 


0 


0 


0 


0 


0 


1 


1 


Total 


146 


574 


409 


228 


93 


8 


2 


1,460 



Wofe; An example can help in interpreting this table. Column 4 indicates that there were originally 228 records with 
four rating categories. After the entries coded 9+ (entries that could not be ordered or that did not reflect performance 
ratings/levels) were deleted, 215 entries with 4 rating categories remained, while 9 of the original 228 were reclassified 
as having three rating categories, and 4 were reduced to zero. No records had seven rating categories originally or after 
data cleaning. 

Source; Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California 
Department of Education. 



in every rating category (58 records). The num- 
ber of performance ratings/levels was reduced 
in 80 records where some but not all of the 
entries were coded with a 9 + (see table B2 for 
a breakdown of recodes). For example, a record 
might have originally included four rating cat- 
egories, but after the deletion of a nonsensical 
category, the number was reduced to three. 
With these classification and cleaning tasks 
completed, 1,402 records remained, including 
35 for which performance ratings/levels were 
entered but no corresponding principal counts. 
These 35 records were not included in the anal- 
ysis of principal rating categories (see table 3 in 
the main report), which left 1,367 records as 
the basis for the analysis of 8,782 principals by 
rating category. 

Challenges and limitations 

This analysis was limited by the content and 
structure of the questions on the survey, as 
well as by the self-reported nature of the data. 
Responses to questions 1.1 and 1.2 (“Which 



best describes the system your LEA uses to eval- 
uate teachers/principals? Select all that apply”) 
were particularly challenging to analyze, for 
three reasons. First, the response choices for 
question 1.1 were not focused on parallel 
aspects of the evaluation systems. The first two 
response choices (“local board-approved” and 
“bargained evaluation system”) were focused 
on the process for developing and adopting 
the evaluation systems, while the third choice, 
“Evaluation system based on the California 
Standards for the Teaching Profession,” was 
related to the standards that underlie the sys- 
tem. (Question 1.2, which, in addition to offer- 
ing the “other” response option, included only 
the first two response choices from 1.1, was less 
of a problem in this regard.) 

Second, the responses submitted for the 
“other” option indicated that respondents 
interpreted the question prompt in broad 
terms. In particular, they commented on 
many different aspects of their evaluation sys- 
tems — on the components, measures, and 
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instruments used in their systems; the aspects 
of the development and adoption process not 
captured in the closed-response items; the pro- 
cess by which the evaluations are conducted; 
the personnel who conduct the evaluations; 
and the basis or foundation for their systems. 
The “other” option acted as a vehicle to capture 
all the issues that the closed responses did not, 
and there was an imbalance in the number of 
responses received through the two response 
formats. Specifically, 143 respondents marked 
“other” in response to question 1.1, and 124 of 
these respondents submitted informative nar- 
rative descriptions elaborating on their “other” 
responses. For the analogous principal evalua- 
tion question (1.2), 578 respondents marked 
“other,” and 477 of these respondents submit- 
ted informative elaborations. This imbalance in 
the number of responses received through the 
two response formats may reflect a respondent 
bias for marking items that were listed as closed 
responses, and thus the data might not fully 
capture all aspects of the evaluation systems. 

Third, the mark-all-that-apply format of 
questions 1.1 and 1.2 rendered interpretation 
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a challenge, especially in light of the many 
resulting possible combinations of responses 
highlighted in table Cl in appendix C. This 
issue is especially salient for item 1.1, which 
has more response choices than 1.2 that are 
not focused on parallel aspects of the evalua- 
tion systems. 

Notwithstanding these issues, the 
responses to questions 1.1 and 1.2 provide 
information that gives a sense of how local edu- 
cation agencies describe their teacher and prin- 
cipal evaluation systems. 

Another overarching issue with the under- 
lying data is that the data were self-reported. 
California Department of Education officials 
did ask respondents to revise uninformative 
responses and fill in missing data (for more 
information about these requests, see the 
“Data” section above). However, no other veri- 
fication process was completed, and no other 
data were analyzed to triangulate the survey 
data for this study. Information reported on 
the survey might be incorrect to the extent that 
the respondents did not understand the ques- 
tions or misreported responses. 
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Appendix C 
Additional data tables 

This appendix includes tables showing cross- 
tabulations of local education agency (LEA) 



response choice combinations for three survey 
questions. 



TABLE Cl 

Cross-tabulation of response choice combinations to question "Which best describes your LEA system of 
teacher evaluation?" [N = 1,482) 



Closed 


District 


Charter 






Total 


response 


Number 


Percent 


Number Percent 


Number 


Percent 


Local board-approved evaluation system 


75 


7.4 


151 


32.2 


226 


15.2 


Bargained evaluation system 


203 


20.0 


14 


3.0 


217 


14.6 


Evaluation system based on the California 
Standards for the Teaching Profession (CSTP) 


208 


20.5 


134 


28.6 


342 


23.1 


Other evaluation system 


11 


1.1 


65 


13.9 


76 


5.1 


Local board-approved evaluation system 
+ Bargained evaluation system 


46 


4.5 


a 


a 


a 


a 


Local board-approved evaluation system 
+ Evaluation system based on the CSTP 


32 


3.2 


35 


7.5 


67 


4.5 


Local board-approved evaluation system 
+ Other evaluation system 


a 


a 


a 


a 


a 


a 


Bargained evaluation system 
+ Evaluation system based on the CSTP 


178 


17.6 


10 


2.1 


188 


12.7 


Bargained evaluation system 
+ Other evaluation system 


5 


0.5 


_ 


_ 


5 


0.3 


Evaluation system based on the CSTP 
+ Other evaluation system 


a 


a 


28 


6.0 


a 


a 


Local board-approved evaluation system 
+ Bargained evaluation system 
+ Evaluation system based on the CSTP 


233 


23.0 


19 


4.1 


252 


17.0 


Local board-approved evaluation system 
+ Bargained evaluation system 
+ Other evaluation system 


a 


a 






a 


a 


Local board-approved evaluation system 
+ Evaluation system based on the CSTP 
+ Other evaluation system 


a 


a 


8 


1.7 


a 


a 


Bargained evaluation system 
+ Evaluation system based on the CSTP 
+ Other evaluation system 


9 


0.9 






9 


0.6 


Local board-approved evaluation system 
+ Bargained evaluation system 
+ Evaluation system based on the CSTP 
+ Other evaluation system 


6 


0.6 






6 


0.4 


Total 


1,013 


68.4 


469 


31.6 


1,482 


100.0 



— is not applicable. 

a. To protect confidentiality, results are not displayed for cells with three or fewer responses orfor cells that could be used to calculate values for such cells. 

Wofe; This table first lists the number and percentage of respondents who indicated single response choices, followed by those who selected each combina- 
tion of two, three, and four response choices, respectively. 

Source; Authors' analysis of data from California Teacher and Principal Evaluation Survey data provided by the California Department of Education. 
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TABLE C2 

Cross-tabulation of response choice combinations to question "Which best 
describes your LEA system of principal evaluation?" 



Closed 


District 


Charter 


Total 


response 


Number Percent 


Number Percent 


Number Percent 


Local board-approved evaluation system 


57.4 


581 


61.8 


290 


58.8 


871 


Bargained evaluation system 


1.6 


16 


a 


a 


a 


a 


Other evaluation system 


31.9 


323 


29.2 


137 


31.0 


460 


Local board-approved evaluation system 
+ Bargained evaluation system 


2.1 


21 


a 


a 


a 


a 


Local board-approved evaluation system 
+ Other evaluation system 


4.1 


42 


3.0 


14 


3.8 


56 


Bargained evaluation system 
+ Other evaluation system 


a 


a 


— 


— 


a 


a 


Total 


68.4 


1,013 


31.6 


469 


100 


1,482 



— is not applicable. 

a. To protect confidentiality, results are not displayed for cells with three or fewer responses. 

/Vote; This table first lists the number and percentage of respondents who indicated single response choices, followed 
by those who selected each combination of two response choices. 

Source.- Authors' analysis of data from California Teacher and Principal Evaluation Survey provided to REL West by CDE. 
TABLE C3 

Cross-tabulation of response choice 
combinations to question "Are 
principal evaluation results publicly 
available?" (N = 1,482) 



1 Category 


Number 


Percent I 


No 


1,379 


93.0 


Yes 


103 


7.0 


Total 


1,482 


100.0 



Source.- Authors' analysis of data from California 
Teacher and Principal Evaluation Survey data pro- 
vided by the California Department of Education. 
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Instructions for completing the 
Teacher and Principal Evaluation 
Survey items. May 25, 2010 

Sub-recipients that received ARRA [American 
Recovery and Reinvestment Act] State Fiscal 
Stabilization Funds [SFSF] must complete the 
survey. 

To Access the Survey 

http://www2.cde.ca.gov/arrareporting/ 

Teacher and Principal Evaluation Survey 

States receiving funds under the SFSF must 
provide assurances in four key areas of educa- 
tion reform. This survey will be used to col- 
lect data required for the first reform area, 
achieving equity in teacher and principal dis- 
tribution. It is important that data be reported 
completely and accurately, as it will be made 
publicly available, as required by statutory 
assurances. 

Teacher information per school 

SFSF indicator (a)(4) requires that each LEA 
[local education agency] indicate the number 
and percentage of teachers rated at each per- 
formance level or rating. To enter this informa- 
tion, begin by selecting “Manage Teacher Cat- 
egories.” Use the “Add a Category” button to 
enter the first rating level of the teacher evalu- 
ation system. For example, this category might 
be named “Satisfactory.” The ranking order 
indicates the level of performance for that 
category. The highest rank, 1, would be used 
to indicate the highest level of performance 
a teacher can demonstrate. Click the “Save 
Changes” button. Continue to add categories 
until the number and descriptions of catego- 
ries exactly matches those used in the teacher 
evaluation system. After saving the changes, 
select the “School Level Data-Teachers” but- 
ton. Enter the total numbers of teachers that 
are rated in each category for each school in 
the LEA. Include all teachers at the site regard- 
less of the last date of evaluation. The sum of 
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all categories should equal the total number of 
teachers at the site. 

Principal information per school 

SFSF indicator (a)(7) requires that each LEA 
indicate the number and percentage of princi- 
pals rated at each performance level or rating. 
To enter this information, begin by selecting 
“Manage Principal Categories.” Use the “Add 
a Category” button to enter the first rating 
level of the teacher evaluation system. For 
example, this category might be named “Satis- 
factory.” The ranking order indicates the level 
of performance for that category. The highest 
rank, 1, would be used to indicate the high- 
est level of performance a principal can dem- 
onstrate. Click the “Save Changes” button. 
Continue to add categories until the number 
and descriptions of categories exactly matches 
those used in the principal evaluation system. 
After saving the changes, select the “School 
Level Data-Principals” button. Enter the num- 
ber of principals in the LEA that are rated in 
each category. 

7.7 Descriptor: (a)(1) 

Which best describes the system your LEA 
uses to evaluate teachers? Select all that apply. 
[Drop down menu] 

• Local board-approved evaluation 
system. 

• Bargained evaluation system. 

• Evaluation system based on the Cali- 
fornia Standards for the Teaching 
Profession. 

• Other evaluation system (describe). 

7.2 Descriptor: (a)(2) 

Which best describes the system your LEA 
uses to evzhinte principals? Select all that apply. 
[Drop down menu] 

• Local board-approved evaluation 
system. 

• Bargained evaluation system. 

• Other (describe). 
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2. 7 Descriptor: (a)(1) 

How are the results of the teacher evaluation 
system used to evaluate: 

a. decisions regarding teacher 
development 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

b. decisions regarding teacher 
compensation 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

c. decisions regarding teacher promotion 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

d. decisions regarding teacher retention 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

e. decisions regarding teacher removal 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

2.2 Descriptor: (a)(2) 

How are the results of the principal evaluation 
system used to evaluate: 

a. decisions regarding principal 
development 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

b. decisions regarding principal 
compensation 

• Not used for this purpose. 



• Used as partial evidence. 

• Used as primary basis for decisions. 

c. decisions regarding principal 
promotion 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

d. decisions regarding principal retention 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

e. decisions regarding principal removal 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

3.1 

How are student achievement outcomes or stu- 
dent growth data used to evaluate the perfor- 
mance of teachers as an evaluation criterion? 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

3.2 

How are student achievement outcomes or stu- 
dent growth data used to evaluate the perfor- 
mance oi principals as an evaluation criterion? 

• Not used for this purpose. 

• Used as partial evidence. 

• Used as primary basis for decisions. 

4.1 

Are principal evaluation results publicly 
available? 

• Yes. 

. No. 
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5.1 

The CDE [California Department of Education] provided the data below for illustrative purposes 
only; they do not represent actual submissions. 

FIGURE D1 

- Teacher Information per School 

Enter the total numbers of teachers that are rated In each category. Include all teachers at the site 
regardless of the last date of evaluation. The total of all categories should equal the total number of 
teachers at the site. If there are no teachers rated In a particular category, please enter 'O'. 



1 School Name 


1 S ati sf acto ryl U n s ati sf acto ry 


Academy of Alameda 


|o 


] 1 


Alameda Fligh 


[37 


] h 1 


Alameda Science and T echnology Institute 1 3 


] |o 1 


Amelia Earhart Elementary 


|s 


] |o 1 


Bay Farm Elementary 


1^ 


] lo 1 


Donald D. Lum Elementary 


h 


] 1 


Edison Elementary 


8 


] h 1 


Encinal Fligh 


14 


] h 1 


Frank Otis Elementary 


2 


] h 1 


Franklin Elementary 


2 


] |o 1 


Henry Haight Elementary 


111 


] |o 1 


Island High (Continuation) 


3 


] h 1 


Lincoln Middle 


11 


] |o 1 


Ruby Bridges Elementary 


|12 


] |o 1 


Washington Elementary 




] |o 1 


Will C. Wood Middle 


12 


] h 1 


William G. Paden Elementary 


ii 


] ^ 1 



[Save I [Cancel | 



5.2 

CDE provided the data below for illustrative purposes only; they do not represent actual 
submissions. 



FIGURE D2 



- Principal Information Districtwide 

Enter the total numbers of principals In the LEA that are rated in each category. Include all principals 
regardless of the last date of evaluation. The total of all categories should equal the total number of 
principals in the district. If there are no principals rated In a particular category, please enter 'O'. 

Practice notji j 

consistent with 
standards: 

Practice thatfTo j 

exemplies 
standards: 

[ Save Principal Irformalicin | | Cancel | 
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Appendix E 

Sample screenshots from the 
California Teacher and Principal 
Evaluation Survey 

The California Department of Education pro- 
vided the data below for illustrative purposes 
only; they do not represent actual submissions 
(figures E1-E3). 



FIGURE El 

Screenshot of California Teacher and Principal Evaluation Survey page for entering 
evaluation system information 



Great Teachers and Principals Survey 

LEA Name: Alameda City Unified 

- Evaluation System Information 



Which best describes the system your LEA uses to evaiuate teachers? (Select all that apply) 

0 Local Board-approved evaluation system 
0 Bargained evaluation system 

0 Evaluation system based on the California Standards for the Teaching Profession 
□ other evaluation system (describe) 

Other teacher evaluation 
description: 




T □ what extent are the results of the teacher evaluation system(s) identified above used to evaluate: 
decisions regarding teacher development? 
decisions regarding teacher compensation? 
decisions regarding teacher promotion? 
decisions regarding teacher retention? 
decisions regarding teacher removal? 

To what extent are student achievement outcomes or I Used as partial evidence 

student growth data used to evaluate the performance of ' 

teachers as an evaulation criterion? 

Which best describes the system your LEA uses to evaluate princpals? (Select all that apply) 

0 Local Board-approved evaluation system 
0 Bargained evaluation system 
□ other evaluation system 

other principal evaluation Li 

description: 



Used as partial evidence 
Select One... 

Not used for this purpose 



Used as partial evidence 



Used as primary basis for decisions 
uisBU ai^'primdiy udsis lur utiumuiis * ' 

I Used as primary basis for decisions 



1 Used as partial evidence 



[ Used as partial evidence 



I Used as partial evidence 



To what extent are the results of the pr/r?cipa/ evaluation system(s) identified above used to evaluate; 
decisions regarding principal development? 
decisions regarding principal compensation? 
decisions regarding principal promotion? 
decisions regarding principal retention? 
decisions regarding principal removal? 

To what extent are student achievement outcomes or 
student growth data used to evaluate the performance of 
principals as an evaulation criterion? 



I Used as primary basis for decisions v 



[ Used as primary basis for decisions ' 



I Used as partial evidence 



Are Principal evaluation results publicly 
available? 



OYes ®No 



Save Evaluation System Information 
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FIGURE E2 

Screenshot of California Teacher and Principal Evaluation Survey page for entering 
teacher information per school 

- Teacher Information per School 

Enter the total numbers of teachers that are rated In each category. Include all teachers at the site 
regardless of the last date of evaluation. The total of all categories should equal the total number of 
teachers at the site. If there are no teachers rated In a particular category, please enter 'O'. 



1 School Name 


1 S ati sf acto ry 1 U n s ati sf acto ry 


Academy of Alameda 


|o 


] h 


:□ 


Alameda Fligh 


[37 


] h 


:□ 


Alameda Science and T echnology Institute j 3 


] |o 




Amelia Earhart Elementary 




] |o 




Bay Farm Elementary 


|7 


] lo 


:□ 1 


Donald D. Lum Elementary 


U 


] h 


:□ 


Edison Elementary 




] h 


:□ ' 


Encinal High 


|l4 


] h 


:□ 


Frank Otis Elementary 


|2 


] h 


:□ : 


Franklin Elementary 


|2 


] |o 


^ 1 


Henry Haight Elementary 


111 


] |o 


^ 1 


Island High (Cohtinuation) 


|3 


] h 


^ I 


Lincoln Middle 


|11 


] lo 


:□ i 


Ruby Bridges Elementary 


|12 


] |o 


^ 1 


Washington Elementary 


U 


] |o 


:□ 


Will C. Wood Middle 


|12 


] h 


:□ 


William G. Paden Elementary 


1^ 


] ^ 


:□ 



[Save I [Cancel | 



FIGURE E3 

Screenshot of California Teacher and Principal Evaluation Survey page for entering 
principal information districtwide 

- Principal Information Districtwide 

Enter the total numbers of principals in the LEA that are rated ih each category. Include all principals 
regardless of the last date of evaluation. The total of all categories should equal the total number of 
principals in the district. If there are no principals rated in a particular category, please enter 'O'. 

Practice nbtji j 

consistent with 
standards; 



Practice thatjio 
exemplies^ 
standards: 



Save Principal Informalion | | Cancel | 
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Letter from the California 
Department of Education requesting 
corrections to the Teacher and 
Principal Evaluation Survey, July 15, 
2010 

Instructions for data correction for the 
State Fiscal Stabilization Fund: Teacher and 
Principal Evaluation Survey 

Overview. You will need to review the data sub- 
mitted for the Teacher and Principal Evalu- 
ation Survey prior to its posting for public 
review on the California Department of Edu- 
cation (CDE) Web site. The data is available at 
https://www2.cde.ca.gov/greatteachers/search. 
aspx. We request that you review the teacher 
and principal rating categories to ensure that 
there are no spelling errors and that all data are 
correct. 

Corrections only need to be made if: 

• There is a data entry error. 

• The total number of teachers that are 
rated in each category for each school 
was not submitted, left blank or is in 
error. 

• The total number of principals in the 
local educational agency (LEA) that 
are rated in each category is not sub- 
mitted, left blank or is in error. 

• There are no teachers/principals rated 
in a particular category, “0” must be 
entered in for that category. No cat- 
egory should be left blank. 

• If the categories under “Teacher Infor- 
mation per School” and “Principal 
Information District wide” are not 
in the ranking order specified in the 
instructions. NOTE: The Highest 
rank, I, must be used to indicate the 
highest level of performance demon- 
strated by a teacher or principal. 
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• There are spelling errors in the rating 
categories. 

PLEASE NOTE: If the total number of 
teachers or principals entered for the LEA is 
less than ten, an asterisk will display in lieu of 
the data to ensure confidentiality. You do not 
need to correct this data. 

Should you need to make any changes, 
please refer to the following instructions: 

To access the survey. Go to http://www2.cde. 
ca.gov/arrareporting. 

• Click on the link that represents your 
LEA (either School District or County 
Office of Education or Direct-funded 
Charter). 

• Locate your County from the drop- 
down list. 

• Locate your LEA from the dropdown 
list. 

• Enter your Entity Password and Sys- 
tem Password. 

• Click submit. 

• Under the Grants heading, click on 
Teacher and Principal Evaluation Survey. 

• To update Teacher Information, scroll 
down to Teacher Information by 
School. 

• To update Principal Information, 
scroll down to Principal Information 
District wide. 

Rating categories and ranking orders 

• Rating categories should reflect the 
same rating terminology used by the 
school in its performance evaluation 
ratings, or a commensurate rating sys- 
tem that conveys the level of perfor- 
mance achieved. 

• A minimum of two ratings categories 
are required. 

• The highest rating must be assigned 
Ranking Order 1, the next highest 2, 
etc. (Refer to examples shown below.) 
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SAMPLE A 


Category Name 


Ranking 

order 


Meets expectations 


1 


Does not meet expectations 


2 



SAMPLES 



Category Name 


Ranking 

order 


Exceeds standards 


1 


Meets standards 


2 


Partially meets standards 


3 


Does not meet standards 


4 



To update teacher categories 

• Click on “Manage Teacher Categories.” 

• Click “Add a Category.” 

• Enter the Category Name. 

• Enter the Ranking Order. 

• Click the “Save Changes” button. 

• Continue to add categories and assign 
ranks as needed. 

• When you have finished click “Done.” 

• To delete categories - Click Delete. 

To update principal categories 

• Click on “Manage Principal Categories.” 

• Click “Add a Category.” 

• Enter the Category Name. 

• Enter the Ranking Order. 

• Click the “Save Changes” button. 

• Continue to add categories and assign 
ranks as needed. 

• When you have finished click “Done.” 

• To delete categories - Click Delete. 

To add or update school level data 

• Select the “School Level Data-Teach- 
ers” button. 

• The screen will display a list of schools 
within your LEA. 



• Enter the total number of teachers 
that are rated in each category for each 
school in the LEA. Include all teachers 
at the site regardless of the last date of 
evaluation. 

• If there are no teachers rated in a cat- 
egory, please enter “0.” If the category 
is blank, the survey will be considered 
incomplete. 

• Click the “Save” button after you com- 
plete this section. 

To add or update district level data 

• Select the “District Level Data-Princi- 
pals” button. 

• The screen will display a list of the rat- 
ing Categories you previously entered. 

• Enter the total numbers of principals 
that are rated in each category. Include 
all principals regardless of the last date 
of evaluation. 

• If there are no principals rated in a cat- 
egory, please enter “0”; if the category 
is blank, the survey will be considered 
incomplete. 

• Click the “Save” button after you com- 
plete this section. 

• There is no submit button on the sur- 
vey. You need to re-save the survey to 
resubmit it. 

Confirmation page 

The confirmation page is available to view what 
you have submitted. 

To access the confirmation page 

• Click on Menu. 

• Scroll down to Grants, Teacher and 
Principal Evaluation Survey. 

• Select “Confirmation Page” Passwords. 

If you have misplaced your password or 

require password assistance, please email arra@ 
cde.ca.gov. Please remember that passwords 
only work when they are copied and pasted to 
the log on page. 



26 





REL Technical Brief REL 2012-No. 023 



Appendix G 



Appendix G 

Letter from the California 
Department of Education requesting 
corrections to the to the Teacher 
and Principal Evaluation Survey, 
September 24, 2010 



OF PUBLIC INSTRUCTION 



Dear County and District Superintendents and Charter School Administrators: 

UPCOMING RELEASE OE 
2010 STATE EISCAL STABILIZATION EUND 
GREAT TEACHER AND PRINCIPAL SURVEY DATA 

This letter is to alert you to the posting of 2010 State Fiscal Stabilization Fund (SFSF) Teacher and 
Principal Evaluation Survey data. 

Posting of 2010 Survey Data 

During the summer of 2010, your local education agency (LEA) submitted data via the SFSF 
Teacher and Principal Evaluation Survey as required by the SFSF assurances. The pre-release of your 
submitted data are available on the California Department of Education (CDE) Web site at https:// 
www2.cde.ca.gov/greatteachers/search.aspx. It is critical that LEAs review these data carefully and 
immediately take steps to remedy any errors, as this data will be publicly available on the CDE Web- 
site on September 30, 2010. The data are embargoed until that time. 

Prior communications regarding the Teacher and Principal Evaluation Survey are available on 
the CDE American Reinvestment and Recovery Act (ARRA) Web page at www.cde.ca.gov/ar/rr/ 
rptingdatcol.asp. 

Corrections to 2010 Teacher and Principal Evaluation Survey Data 

Two actions must be taken prior to September 29, 2010, to correct Teacher and Principal Survey 
data: 

1. The Teacher and Principal Evaluation Survey must be resubmitted electronically via the 
ARRA Federal Reporting page on the CDE Web site located at http://www2.cde.ca.gov/ 
arrareporting/. Detailed instructions for completing this task are included in the attach- 
ment to this email. 

2. An email must be sent to the Title II Leadership Office verifying resubmission. 




September 24, 2010 



JACK O'CONNELL 

STATE SUPERINTENDENT 



CALIFORNIA 
DEPARTMENT OF 
EDUCATION 



1430 N STREET, SACRAMENTO. CA 93814-5901 • 916-319 0800 • WWW.CDE CA GOV 
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Excerpts from Federal Register and 
specific State Fiscal Stabilization 
Funds report requirements 
pertaining to the California Teacher 
and Principal Evaluation Survey 
FRDoc E9-17906 [Federal Register: July 29, 
2009 (Volume 74, Number 144)] 

[Notices] 

[Page 37837-37872] 

From “Achieving Equity in Teacher Distribu- 
tion” section 

. . . With respect to evaluation of teacher per- 
formance, we propose to require that a State 
provide descriptive information on the teacher 
performance evaluation systems used in local 
educational agencies (LEAs) in the State, 
including an indication of whether any official 
systems used to evaluate teacher performance 
include student achievement outcomes as an 
evaluation criterion. With respect to teacher 
performance ratings or levels, we propose to 
require that a State provide data on the dis- 
tribution of performance ratings or levels in 
its LEAs as well as an indication of whether 
such ratings or levels are available to the pub- 
lic by school for each LEA. When properly 
developed and implemented, local evaluation 
systems perform a principal role in measuring 
teacher effectiveness. We also believe that stu- 
dent achievement outcomes are a central factor 
in evaluation systems that yield fair and reli- 
able assessments of teacher performance. The 
data and information on teacher performance 
ratings or levels, together with the descriptive 
information on teacher performance evalua- 
tion systems, will provide greater transparency 
on the design and usage of teacher evaluation 
systems and will serve as an important indica- 
tor of the extent to which effective teachers are 
equitably distributed within LEAs and States. 



Moreover, this information will help States and 
other stakeholders correct inequities in the dis- 
tribution of effective teachers as well as short- 
comings in the design and usage of teacher 
performance evaluation systems . . . regarding 
evaluation of principal performance, we pro- 
pose requirements similar to those proposed 
for evaluation of teacher performance, except 
that we do not propose to require a State to 
indicate whether principal performance ratings 
or levels are available to the public by school in 
each LEA, as such information may be person- 
ally identifiable. 

In order to meet the proposed requirements to 
describe the teacher and principal performance 
evaluation systems used in LEAs in the State, 
a State would not be required itself to develop 
such descriptions; it would be sufficient for 
the State to maintain a Web site that contains 
electronic links to descriptions developed by 
its LEAs. On such a Web site, the State could 
also include, by LEA, the data and information 
the State collects in order to meet the other 
proposed requirements that relate to evalu- 
ation of teacher and principal performance 
(i.e., the requirements to indicate whether offi- 
cial teacher and principal evaluations systems 
include student achievement outcomes as an 
evaluation criterion, to provide the number 
and percentage of teachers and principals rated 
at each performance rating or level in official 
evaluation systems, and to indicate whether 
the number and percentage of teachers rated 
at each performance rating or level in official 
evaluations systems are publicly available for 
each school). In such a case, however, the State 
would be responsible for ensuring, through 
appropriate guidance or technical assistance, 
that the descriptions of teacher and principal 
performance evaluation systems maintained by 
LEAs contain the required information and are 
provided in an easily understandable format. 
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Specific State Fiscal Stabilization Funds 
reporting requirements pertaining to the 
California Teacher and Principal Evaluation 
Survey 

The below indicators are summarized at www. 
cde.ca.gov/ar/sf/sfsfphase2id.asp. 

Descriptor (a)(1) 

Describe, for each local educational agency 
(LEA) in the State, the systems used to evalu- 
ate the performance of teachers and the use of 
results from those systems in decisions regard- 
ing teacher development, compensation, pro- 
motion, retention, and removal. 

Indicator (a)(3) 

Indicate, for each LEA in the State, whether 
the systems used to evaluate the performance 
of teachers include student achievement out- 
comes or student growth data as an evaluation 
criterion. 

Indicator (a)(4) 

Provide, for each LEA in the State whose 
teachers receive performance ratings or levels 
through an evaluation system, the number and 
percentage (including numerator and denomi- 
nator) of teachers rated at each performance 
rating or level. 



Appendix H 



Indicator (a)(5) 

Indicate, for each LEA in the State whose 
teachers receive performance ratings or levels 
through an evaluation system, whether the 
number and percentage (including numerator 
and denominator) of teachers rated at each per- 
formance rating or level are publicly reported 
for each school in the LEA. 

Descriptor (a)(2) 

Describe, for each LEA in the State, the sys- 
tems used to evaluate the performance of prin- 
cipals and the use of results from those systems 
in decisions regarding principal development, 
compensation, promotion, retention, and 
removal. 

Indicator (a)(6) 

Indicate, for each LEA in the State, whether 
the systems used to evaluate the performance 
of principals include student achievement out- 
comes or student growth data as an evaluation 
criterion. 

Indicator (a)(7) 

Provide, for each LEA in the State whose prin- 
cipals receive performance ratings or levels 
through an evaluation system, the number and 
percentage (including numerator and denomi- 
nator) of principals rated at each performance 
rating or level. 
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Notes 

1. California has three types of local edu- 
cation agencies: school districts, direct- 
funded charter schools, and county 
offices of education. 

2. California has three types of local 
education agencies: school districts, 
direct-funded charter schools, and 
county offices of education. Because 
the unit of analysis for this study is the 
evaluation system rather than the local 
education agency, and comparisons 
among these systems can yield relevant 
policy insights, school districts and the 
state’s 58 county offices are considered 
together. Both manage schools and 
programs (for example, California’s 
county offices manage the state’s 283 
community day schools for high-risk 
students and 42 Regional Occupa- 
tional Centers), and both evaluate the 
teachers working in them. 

3. Direct-funded charter schools refer to 
schools funded directly by the state 
of California rather than through 
districts or county offices. Each Cali- 
fornia charter school may elect to 
receive its funding directly (in lieu of 
having it disbursed to the local educa- 
tion agency that granted its charter) 
by notifying its local county super- 
intendent. According to the Califor- 
nia Department of Education, 71 per- 
cent of charter schools in California 
are directly funded. The 469 direct- 
funded charter schools in this study 
self-identified as such when completing 
the survey. District- or county-funded 
charter schools are included with local 
education agencies because their sur- 
veys were submitted by local educa- 
tion agencies and the results were not 
generally broken down by school, so 
no data specific to district- or county- 
funded charter schools were available. 



Notes 



4. An evaluation system is a system devel- 
oped through negotiations between 
district management and a local labor 
union. 

5. ILDI members include the California 
Department of Education, the Com- 
mission on Teacher Credentialing, 
county offices of education, public and 
private universities, the Association 
of California School Administrators, 
the Center for the Future of Teach- 
ing and Learning, and the California 
Comprehensive Center and Regional 
Educational Laboratory (REL) West 
at WestEd. REL West’s role is primar- 
ily in needs analysis related to leader- 
ship issues in California. Since August 
2008, a representative of REL West 
has attended monthly ILDI meetings 
on policy related to school leaders in 
California. The group has in turn pro- 
vided feedback on REL West projects, 
particularly recent analyses of labor 
market trends for teachers and admin- 
istrators (for example. White, Fong, 
and Makkonen 2010). 

6. Although districts include charter 
schools that are administered by the 
district, the unit of analysis here is 
the evaluation system, so the analy- 
sis focused on local education agency 
responses about their systems: districts 
(including county education offices) 
and direct-funded charter schools. 
Significance tests to test the effects of 
sampling error were not performed, 
since the study data are from 99.5 per- 
cent of the population of local educa- 
tion agencies in California. 

7. Note that, for this analysis, the first 
step was verifying that the perfor- 
mance ratings/levels were entered in 
the order specified in the instructions 
on the survey, whereby a “1” represents 
the highest rating. If any entries were 
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Notes 



clearly out of order — for example, 
“unsatisfactory” is in the slot for the 
highest rank and “satisfactory” is in the 
slot for the next highest rank — they 
were reordered. If any entries with two 
or more levels clearly did not follow an 
ordinal logic, or if the order could not 
be determined because the ratings/lev- 
els were not interpretable, those entries 
were excluded from the analysis. 

8. Such results were evident even though 
state law requires certificated person- 
nel to be evaluated based on “the prog- 
ress of pupils towards the standards 
pursuant to . . . standards of expected 
pupil achievement at each grade level 
in each area of study.” Survey respon- 
dents may have broadly interpreted 
the terms that appeared on the survey, 
however, such as “student achievement 
outcomes or student growth data.” 
Moreover, they could have interpreted 
the phrase “used as an evaluation crite- 
rion” to mean used in either formative 
or summative evaluation (and used to 
varying degrees). 

9. For this section, searches of the Edu- 
cation Resources Information Center 
and ProQuest Education, Psychology, 
and Social Science journal databases 
were limited to peer-reviewed lit- 
erature since 2000 and combined the 
terms “teacher evaluation” or “princi- 
pal evaluation” separately with each of 
the terms “development,” “dismissal,” 
“student achievement,” “value-added,” 
“measures,” “system,” “categories,” “rat- 
ings,” and “district.” 

10. The study was based on analysis of 
teacher demographic and evaluation 
data along with collective bargaining 
agreements in 12 districts in four states: 
Arkansas (El Dorado, Jonesboro, Little 
Rock, Springdale), Colorado (Denver, 
Pueblo), Illinois (Chicago, Rockford, 



District U-46), and Ohio (Akron, 
Cincinnati, and Toledo). The authors 
also collected and analyzed surveys 
from about 15,000 teachers and 1,300 
administrators in those districts 
(response rates for the surveys were not 
reported) and conducted 130 inter- 
views with district personnel to better 
understand their findings. 

11. Goldring et al. (2009) conducted 
a content analysis of 65 evaluation 
instruments across the United States. 

12. Surveys were sent to all school dis- 
tricts in the United States with more 
than 10,000 students (ji = 867). The 
response rate was 22 percent (« = 193). 

13. California Education Code section 
44662 states that “the governing board 
of each school district shall evaluate 
and assess certificated employee per- 
formance as it reasonably relates to 
(1) The progress of pupils toward the 
standards established pursuant to . . . 
standards of expected pupil achieve- 
ment at each grade level in each area 
of study . . . and, if applicable, the state 
adopted academic content standards 
as measured by state adopted criterion 
referenced assessments.” 

14. For this section, searches of the Edu- 
cation Resources Information Center 
and ProQuest Education, Psychology, 
and Social Science journal databases 
were limited to peer-reviewed lit- 
erature since 2000 and combined the 
terms “teacher evaluation” or “princi- 
pal evaluation” and “teacher effective- 
ness” separately with “California.” 

15. The New Teacher Project adminis- 
tered an online survey to all teachers 
in the Los Angeles Unified School 
District (34,956), with a 10 percent 
response rate (3,663 teachers). 

16. For this section, searches of the Edu- 
cation Resources Information Center 
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Notes 



and ProQuest Education, Psychol- 
ogy, and Social Science journal data- 
bases were limited to peer-reviewed 
literature since 2000 and combined 
the search terms “charter school” and 
“teacher evaluation.” 

17. Podgursky and Ballou administered 
the survey to 200 charter schools and 
received 132 completed surveys from 
schools in Arizona, California, Colo- 
rado, Florida, Massachusetts, Michi- 
gan, and Texas. 

18. The authors interviewed teachers 
and principals in the charter schools 
and senior managers in the charter 



management organizations, asking: 
Does teacher evaluation in charter 
schools improve instruction, student 
learning, and achievement? Do charter 
school evaluations exhibit greater vari- 
ation in teacher ratings? Are charter 
school administrators able to use the 
results to identify and dismiss teachers 
who are not effective and reward those 
who are? 

19. County-district-school codes are 
14-digit codes used by the Califor- 
nia Department of Education and 
other state agencies to identify schools 
within California. 
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